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MASSACHUSETTS 


ABSTRACT 


An  asymptotic  analysis  is  carried  out  for  an  approximate  method  of  estimating 
the  parameters  of  the  power  spectrum  of  a  zero -mean  stationary  Gaussian  random 
process  from  an  observed  realization  of  limited  duration.  Maximum  likelihood  esti¬ 
mates  are  obtained  with  the  approximation  that  the  coefficients  of  the  Fourier  series 
expansion  of  the  realization  are  uncorrelated.  This  is  equivalent  to  other  approxima¬ 
tion  techniques  which  assume  a  periodic  covariance  function.  The  dispersion  of  the 
estimates  is  evaluated  in  terms  of  a  quantity  called  the  differential  variance.  It  is 
shown  that  with  this  quantity  as  a  criterion,  the  approximate  estimates  are  as  good, 
asymptotically,  as  the  exact  maximum  likelihood  estimates.  An  approximate  expres¬ 
sion  for  the  differential  variance  in  terms  of  the  power  spectrum  is  given  and  it  is 
shown  that  this  expression  asymptotically  approaches  its  exact  value. 

These  results  follow  from  a  general  expression,  obtained  by  means  of  a  con¬ 
verse  to  the  Schwarz  inequality,  which  compares  the  differential  variance  of  the 
approximate  estimates  with  that  of  the  maximum  likelihood  estimates.  This  expres¬ 
sion  is  evaluated  for  the  power  spectrum  parameter  estimation  problem  in  terms  of 
the  covariance  matrix  of  the  Fourier  coefficients.  The  asymptotic  behavior  of  these 
covariances  is  bounded  so  that  the  convergence  of  the  elements  of  the  inverse  covari¬ 
ance  matrix  can  be  demonstrated.  The  results  on  the  differential  variance  of  the 
approximate  estimates  are  then  established  by  matrix  methods. 
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I.  INTRODUCTION 

Tliis  report  investigates  an  approximate  method  for  estimating  parameters  of 
the  power  spectrum  of  a  zero-mean  stationary  Gaussian  random  process.  Applications 
and  previous  work  are  discussed  in  references  1-4.  The  method  is  based  on  the  ap¬ 
proximation  that  the  Fourier  coefficients  of  a  realization  of  long-time  duration  are  un¬ 
correlated.  By  a  result  of  Root  and  Pitcher^  this  is  equivalent  to  the  approximation 

that  the  covariance  function  of  the  process  is  periodic.  This  latter  approximation  has 
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l>een  used  without  quantitative  justification  by  a  number  of  previous  authors.  The 
burden  of  the  present  report  is  to  justify  the  approximate  method  by  showing  that,  in 
a  certain  asymptotic  sense,  it  provides  estimates  which  are  as  good  as  the  exact 
maximum  likelihood  estimates.  This  is  the  first  time,  to  the  author’s  knowledge,  that 
such  a  quantitative  evaluation  of  the  asymptotic  behavior  of  these  techniques  has 
appeared.  The  arguments  required  have  turned  out  to  be  somewhat  involved,  but  this 
is  not  surprising  in  view  of  the  general  difficulty  of  treating  exactly  problems  con¬ 
cerning  stationary  non -white  Gaussian  processes  over  a  finite  time  interval.  Only  the 
estimation  of  a  single  parameter  has  been  dealt  with  here,  but  the  multi-parameter 
case  can  be  handled  at  the  cost  of  still  further  complexity.  The  methods  can  also  be 
applied  to  the  estimation  of  the  parameters  of  a  deterministic  signal  in  additive 
Gaussian  noise  and  to  corresponding  detection  problems. 

In  Section  II  some  general  results  on  maximum  likelihood  estimates,  the 
Cramer-Rao  lower  bound  and  the  differential  variance  are  discussed.  These  are  ap¬ 
plied  to  power  spectrum  parameter  estimation  in  Section  III.  Section  IV  describes 
some  practical  approximations  to  the  estimates  and  the  differential  variance.  Some 
results  are  established  in  Section  V  for  the  convergence  of  the  covariances  of  the 
Fourier  coefficients  and  in  Section  VI  for  the  inverse  of  the  covariance  matrix. 

Finally,  Section  VII  proves  that  the  differential  variance  of  the  approximate  estimates 
is  as  small  asymptotically  as  that  of  the  exact  maximum  likelihood  estimates  while 
Section  VIII  establishes  the  validity  of  the  approximation  to  the  differential  variance. 
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U.  MAXIMUM  LIKELIHOOD  ESTIMATES  AND  THE  DIFFERENTIAL  VARIANCE 

Some  general  results  of  estimation  theory, are  presented  in  this  Section. 
Consider  an  N-dimensional  vector  random  variable  £with  probability  density  f(x;a) 
depending  on  a  parameter  a  with  true  value  a^.  The  following  discussion  assumes 
that  certain  general  regularity  conditions  on  f(x;a)  are  fulfilled.  For  an  observation 
of  the  log  likelihood  of  a  is  defined  as 

A(i;a)  =  In  f(|;a) 


The  maximum  likelihood  (ML)  estimate  a  is  the  value  of  a  which  maximizes  A(£;a).  It 
can  sometimes  be  found  explicitly  as  the  root  of  the  likelihood  equation 


^  A(i;a)  =  0 


(2-1) 


but  usually  a  linearization  in  the  vicinity  of  or  a  method  of  successive  approxima¬ 
tions  is  required. 

The  Cramer-Rao  lower  bound states  that  for  any  estimate  a* 


Var  a*  2= 


[  1  -h  db/dop]^ 


(2-2) 


where 


b  = 


is  the  bias  of  a*.  An  alternate  expression  for  the  denominator  of  (2-2)  is  obtained 
from  the  identity 

E[A'(|;ao)]^  =  “EA^Oo)  (2-3) 


In  this  report  a  prime  always  denotes  differentiation  with  respect  to  a  and 
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a  -  Qq 


a  =  a. 


"o 


An  unbiased  estimate  whose  variance  satisfies  (2-2)  with  the  equals  sign  is 
said  to  be  efficient-  However,  efficient  estimates  exist  only  in  certain  cases.  ML 
estimates  are  not  necessarily  unbiased  or  efficient. 

Frequently  the  bias  of  an  estimate  under  consideration  cannot  be  established 
so  the  expression  (2-2)  is  not  informative.  However,  the  Cramer-Rao  lower  bound 
can  still  be  interpreted  in  terms  of  the  "sensitivity"  introduced  by  Kelly,  Lyons  and 
Root^^  Tliey  consider  any  statistic  a*  which  is  a  measure  of  in  the  sense  that  its 
expectation  is  a  monotonic  function  of  and  define 


dEg* 

dOo 


sensitivity  = 


standard  deviation  of  g*  for  g  =  g 


'o 


The  reciprocal  of  the  sensitivity  is  just  that  small  change  in  g^  required  to  change  the 
mean  value  of  g*  by  one  standard  deviation.  It  is  seen  that  (2-2)  can  now  be  written 


(2-4) 


Next  consider  the  general  class  of  estimates  obtained  by  maximizing  over  g 
some  function  L(^;g)  which  depends  on  £and  g  (but  not,  of  course,  on  g^)  and  has  the 
further  property 


=  0 


(2-5) 
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The  dispersion  of  the  sampling  errors  can  be  characterized  by  a  quantity  which  may 
be  termed  the  differential  variance. 


E[L'(|;ao)]2 

S(a*)  =  - ^ 

IE  L*’(i;ao)]2 

Tliis  quantity  can  be  further  interpreted  when  L(£;a)  is  sufficiently  regular  so  that  in 
the  vicinity  of  the  true  parameter  value  a^,  it  can  be  approximated  by  the  first  three 
terms  of  the  Taylor  series  expansion 

(a-a  )2 

L(i;a)  «  L(i;ao)+  (a-Oo)  ^  L"(£;ao)  (2-6) 

The  maximum  value  of  L(£;  a)  over  a  occurs  where 

L’(i;a)  =  0  (2-7) 


so  from  (2-6) 


L’(i;ao) 

L'’(i;ao) 


(2-8) 


The  distribution  of  the  sampling  error,  a*  —  a^,  is  approximately  the  same  as  that  of 
the  expression  (2-8).  If  it  is  also  assumed  that  the  random  variations  of  I-."(^;aQ) 
are  small  compared  with  EL"(£;aQ)  then  I-'’'(^;o[q)  can  be  approximated  by  its  expected 
value,  the  random  fluctuation  of  L'(£;aQ)  about  zero  may  be  thought  of  as  a  linear 
noise  term  displacing  the  peak  of  the  parabolic  approximation  to  L(£;a),  and 

Var  a*  «  S(a*)  (2-9) 


4 


12 

It  has  been  shown  by  Godambe  that  if  S(a*)  is  taken  as  a  measure  of  the 
dispersion  of  the  estimate  a*,  without  necessarily  referring  to  the  particular  interpre¬ 
tation  mentioned  above,  then  under  general  regularity  conditions  S(a*)  is  minimized 
when  L(^;a)  =  A(§;a)  and  a*  =  d.  This  minimum  value  is,  by  (2-3), 


S(a) 


E[A'(i;ao)]^ 

lEA”(£;ao)]2 


1 

E[A'(l;a^) 


-  1 

E  A-d.-a^) 


(2-10) 


This  is  the  same  as  the  quantity  appearing  in  the  Cramer -Rao  lower  bound  (2-2),  but 
the  present  result  holds  with  the  equals  sign  for  any  regular  estimate,  biased  or  not. 

We  now  determine  an  upper  bound  for  S(a*)  in  terms  of  the  difference  between 
A(§;a)  and  L(£;a).  By  (2-5)* 


J  L’(x;aQ)  f(x;aQ)  dx  =  0 


(2-11) 


The  differentiation  of  (2-11)  under  the  integral  sign  with  respect  to  gives 


J*  L"(x;aQ)  f(x;aQ)  dx  =  -J  L’(x;  o^)  f'(x;  a^)  dx 

=  -J  L'(x;a  )  J~’  f(x;a  )  dx 

-  o'  f(x;ao)  -  o'  - 


(2-12) 
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There  is  a  modification  of  the  Schwarz  inequality  which  states  that  for  any 
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non-negative  cp(z)  and  for  integrable  u  (z)  cp(z)  and  v  (z)  cp(z) 


( J  u(z)  v(z)  cp(z)  dz  s 


1  - 


J  [u(z)-v(z)]2  cp(z)dz 
J  u2(z)  cp(z)  dz 


J*  u^(z)  cp(z)  dz  J  v^(z)  cp(z)  dz 


(2-13) 


*  Unless  otherwise  noted,  all  integrals  are  N-fold  definite  integrals,  each  component 
of  X  ranging  over  (— ®,  ®). 
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Let  us  assume  the  required  conditions  are  satisfied  so  that  (2-13)  can  be  applied 
to  the  right-hand  side  of  (2-12)  giving 


«  ,  ,  f(x;ao)  2 

[  J  L  (x ;  Oo)  f/::.  .  ,  f(x;  a^)  dx  ] 


f(x;  %) 


'-'  O' 


'  J  {  L* (x;  a  )  -  ( f' (x;  aQ)/f(x;  a  )  ]  }  f(x;  a  )  dx  i 
>  <  1  _  - 2 - > 


J  lL'(x;aQ)l  f(x;aQ)dx 


L  2  rf’(x;ao)  "1^ 

-7  [  L* (x;  a^)  ]  f(x ;  a^)  dx  J  |  ^  y  |  f(x;  a^)  dx 


-  -  o  - 


(2-14) 


Noting  that 


f*(x;go) 
f(x;  a^) 


A’(x;a) 


a  =  a 
o 


and  substituting  (2-12)  into  (2-14)  we  can  rearrange  the  result  as 

J  [L’(x;a^,)]^  f(x;ao)dx  „  I  J  [L’(x;a  -  A'(x;aQ)  f(x;a  )  dx 

•  E(A*(§;a^j)r  1 - ^ ^ ^ ^ - 


-1 


[J  L"(x:aQ)  f(x:a^)dx]‘‘ 


J  (L’(x;aQ)]  f(x:%)dx 

(2-15) 


The  left-hand  side  of  (2-15)  is  just  S(a*)/S(a).  The  right-hand  side  consists  of  a  quantity 
(1  —  C)  ^  If  C  can  be  shown  to  converge  to  zero,  then  it  is  established  that  S(a*) 

converges  to  its  minimum  value  S(a).  This  is  a  tractable  criterion  for  establishing  the 
asymptotic  behavior  of  a  general  class  of  estimates  and  is  applied  in  Section  VII  to  the 
approximate  ML  power  spectrum  parameter  estimates. 
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III.  MAXIMUM  LIKELIHOOD  ESTIMATES  FOR  POWER  SPECTRUM  PARAMETERS 
T!ie  analysis  is  based  on  the  following  assumptions: 

a)  The  random  process  is  stationary,  Gaussian,  and  zero-mean  with  a  double¬ 
sided  power  spectrum  P(f;a)  which  is  a  known  function  of  the  parameter  a  whose  value 
is  to  be  estimated.  (If  the  process  consists  of  a  random  signal  plus  an  independent 
noise,  then  P(f:a)  is  the  sum  of  the  two  individual  spectra. ) 

b)  A  particular  realization  x(t)  of  the  process  is  observed  for  0  ^  t  ^  T. 

c)  The  true  value  of  a  is  denoted  by  and  the  actual  power  spectrum  is 
i({)  =  P{f;a^).  is  assumed  to  lie  within  a  known  finite  interval 

a  .  <  a  <  a 
min  o  max 

(d)  For  any  a  within  this  interval,  only  values  of  P(f;a)  within  some  fixed  finite 

range 

0<  fj<  |f|  <  f2<“ 

depend  on  a.  Since  f^  and  may  be  anywhere  outside  of  the  range  of  dependence  for 
convenience,  we  take 

f,  =  Nj/T  =  N^/T 

where  and  are  integers.  (This  assumption  means  that  P(f;a)  has  some  constant 
shape  outside  of  f  ^  <  |f|  <  The  assumption  1^2  allows  a  finite -dimensional  formu¬ 
lation  of  the  problem  and  is  meaningful  in  the  usual  practical  situation  where  frequencies 
above  some  finite  limit  are  not  observable.  The  assumption  f ^  >  0  is  only  for  the  con¬ 
venience  of  eliminating  zero  frequency  which  enters  unsymmetrically  into  the  formulation. ) 
e)  P(f;a)  obeys  certain  regularity  conditions  which  will  be  detailed  in  Section  V. 

The  Karhunen-Loeve  expansion  which  is  frequently  employed  in  the  analysis  of 
random  processes  is  not  well  adapted  to  spectral  parameter  estimation  problems 
since  it  is  in  terms  of  an  uncorrelated  set  of  eigenfunctions  which,  except  in  the 
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simplest  cases,  change  in  a  complicated  way  as  the  parameters  vary.  Instead,  the 
likelihood  function  is  here  expressed  in  terms  of  the  Fourier  coefficients 


Y  =  ~="  r  x(t)  exp[— i2nnf  t]  dt  (3-1) 

n  Vt  0  ° 

where 

f  =  1/T 
0 

1 3 

The  coefficients  for  ^  n  ^  are  taken  as  the  "observable  coordinates"  of  the 
process.  This  is  plausible  since  under  general  conditions,  for  almost  every  realiza¬ 
tion,  x(t)  can  be  represented  by  the  limit-in-the-mean  of  its  Fourier  series  expansion^ 
This  choice  of  the  observable  coordinates  could  be  further  justified  but  that  will  not  be 
attempted  here. 

The  real  and  imaginary  parts  of  each  have  a  Gaussian  distribution  with  zero- 
mean  and  the  entire  set  of 


N  =  N^-N^+l  =  (f2-fj)T+l  ^  f^T 


(3-2) 


coefficients  has  an  N-variate  complex  Gaussian  distribution.  It  can  be  shown  that 

.  2 

^  r  df  $(f) 

rr^T 


I 

r  =  Ey  Y  =  — T"  f  df  $(f)  77 — — 7=7  (3-3) 

nm  m  '^m  ^2^  J  '  (f  —  n/T)  (f  -  m/T)  '  ' 


where  y^  is  the  complex  conjugate  of  y^.  (3-3)  is  equivalent  to  a  result  in  Reference 
5  except  for  a  correction  of  a  factor  of  two.  The  boundedness  of  §(f)  assures  that  the 
^nm  the  normalization  in  (3-1)  is  chosen  so  that  E|yjj|  has  a  finite  limit  as 

T-  ®. 

With  R  as  the  matrix  having  elements  rjjj^^  and  y  as  the  vector  with  elements 
Yjj,  the  log  likelihood  of  the  parameter  a  is 
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(3-4) 


A(Y;a)  =  -  {In  nN|R|  +  R"^ 


which  is  maximized  over  a  to  determine  a.  It  is  shown  in  Appendix  A  that  R  is  non¬ 
singular  if 


#(f)  >  0  for  all  f 


Next,  let  D  be  the  matrix  with  elements 


‘^nm  da  ^2^  JdfP'dja)  (f  _  ^/T)  (f  -  m/T) 


and  assume  all  d^^^  exist.  Using  matrix  relations  given  by  Bodewig^^  we  have 


^  In  |r|  =  tr  r'^  D 
da  -  - 


(3-6) 


D  R 


-1 


(3-7) 


and  thus 


A’(i;a)  =  -{trR"^  D  -  i^R'^  D  r'^ 


(3-8) 


(In  certain  cases  an  exact  expression  for  a  can  be  found  by  using  (3-8)  in  (2-1). )  The 
fourth  product  moment  for  complex  Gaussian  variates  is 

E  A  ^)  (^^  B_y)  =  trARtrBR+trARBR  (3-9) 

and  therefore 

=  tr  R‘^  D  r“^  D  (3-10) 

which  provides  the  Cramer -Rao  lower  bound  and  the  value  of  S(a). 
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IV.  APPROXIMATE  MAXIMUM  LIKELIHOOD  ESTIMATES  FOR  POWER  SPECTRUM 
PARAMETERS 

Unfortunately,  the  exact  expression  (3-4)  for  ^(Y,a)  is  too  complicated  to  permit 
calculations  in  most  cases.  However,  it  is  easily  seen  that 


lim 

T-oo 


r 

nm 


$(nfo) 

0 


n  =  m 


n  5^  m 


(4-1) 


A  great  simplification  occurs  if  the  approximation  is  made  that  for  large  T 


r 

nm 


0 


n  =  m 
n  7^  m 


(4-2) 


Then  R  becomes  diagonal  and  we  take 


My;  a.)  «  L(Y:a) 


■^2 

-  Tj  I  In  n  P(nfQ;a)  + 
n=N^ 


P(nfQ:a) 


] 


(4-3) 


Thus  the  likelihood  is  approximated  in  terms  of  the  elements  of  the  periodogram 
a*  will  henceforth  denote  the  approximate  ML  estimates  obtained  by  maximizing 
L(Y;ot). 

If  P(f;a)  is  a  slowly  varying  function,  then  the  summation  (4-3)  can  be  approxi¬ 
mated  by  an  integral.  The  individual  Iy^^I^  need  not  be  measured  since  their  values  can 
be  smoothed  and  replaced  in  this  integral  by  a  continuous  function  §*(f)  which  is  equi¬ 
valent  to  an  approximately  unbiased  overall  spectral  estimate  such  as  given  by  the 
Blackman-Tukey  method  '  or  by  a  power  spectrum  analyzer.  For  the  integral  to  be  a 
good  approximation  all  the  IyuP  must  be  effectively  included  and  the  resolution  must 
be  sufficient  so  that  the  detailed  structure  of  the  spectrum  is  not  obscured.  With  (4-2), 
IYj^I  is  replaced  by  $*(nfQ)  in  (4-3)  and 

L(x;  a)  «  -  T  J*  df  I  log  TT  p(f:  a)  +  ]  (4-4) 
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where  the  integral  is  multiplied  by  T,  the  reciprocal  of  the  spacing  of  the  terms  in  the 
summation. 

It  is  now  readily  seen  that 


l!.(r.a)  =  - 


P*(nfo;a) 

P(nfQ;a) 


IVnP  P’(nfo;a)-| 
P2(nf  ;a)  J 

o 


(4-5) 


and  that 


E  L’(Y;a^)  =  0 


(4-6) 


which  satisfies  (2-5).  A  second  differentiation  and  the  substitution  of  (4-2)  give 


-  E  L'’(y;o[<j) 


P*(f;ao)~|' 
P(f;ao)  _ 


(4-7) 


With  the  discussion  in  Appendix  B  and  (3-2),  it  can  readily  be  established  that 
E  L”(Y;aQ)  is  of  the  order  of  T  while  the  absolute  value  of  the  error  in  approximating 
the  sum  by  the  integral  is  bounded  by  a  constant.  By  some  further  calculations  of  this 
sort  the  approximation  introduced  just  before  (2-9)  can  also  be  justified  for  this  case. 

We  will  take  —  E  L”(^;aQ)  as  a  convenient  approximation  to  —  E  A"(^;aQ),  the 
quantity  entering  into  (2-2)  and  (2-10).  It  should  be  noted  that 

E(L*(x:aQ)]^  5^  -E  L’’(^;ao) 


because  the  Yj^  are  not,  in  general,  uncorrelated. 
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V.  CONVERGENCE  OF  THE  COVARIANCES  OF  THE  FOURIER  COEFFICIENTS 

For  our  purposes  it  is  necessary  to  show  that  for  some  T^  >  0  and  T  >  T^  the 
covariances  r^^j^  converge  uniformly  and  with  sufficient  rapidity  to  their  asymptotic 
values  for  all  n,m  satisfying 


Tf|  n  s  Tf2 

Tfj  <  m  <  Tf2 


(5-1) 


This  can  be  established  with  the  assumptions  on  P(f;a)  given  in  Section  III  plus  the  one 
below. 


(e)  For  all  f  and  all  a  within  a  .  <  a<  a  there  exist  finite  constants  P  , 

mm  max  max 

Q  .  P'  ,  Q'  >  Prnin  that 

max  max  max  min 


P(f;a)  £  P 


max 


5P(f;a) 

Sf 


max 


5P(f;a) 

3a 

3f  3a 


^  P’ 


max 


P(f;a) 


^  Q’. 


max 


P(f;a)  ^ 


P  .  >  0 

min 


and  all  indicated  derivatives  exist  and  are  continuous. 

The  variances  r  are  considered  first.  From  (3-3) 
nn 


nn 


-r2'r 

TT  T  —  00 


r  df  $(f) 


sin  TTfT 
(f-n/T)2 


1  r  ir  s/r  .  sin  tti  T 

df?(f  +  n/T)  - 3 - 

1 


00 

00 


tt2t  _ 


J  df  ($(n/T)  +  $(f  +  n/T)  -  5(n/T)  ] 


sin  nfT 
f2 


(5-2) 
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Since 


tt^T  _ 


f  df  =  1 

CO  4 


(5-3) 


then 


V  -  =  Sn 


^2  -  ¥2  “ 


l-2{^  -CO  2f2 


By  the  mean  value  theorem  for  the  derivative 
|$(f+n/T)-$(n/T)|  ^  |f|  Q 


J  +  J  +J  i  j  df  [$(f  +  n/T)-$(n/T)] 


sin^TTfT 

f2 

(5-4) 


max 


(5-5) 


so 


[^2  sin^nfT 

J  ^^Qmax  Ifl 

(  -212 

00 

"  ^  i 

’--2f2  '  ' 

2^2  1 

df  P 


sm 


^i] 


(5-6) 


With  the  substitution 


u  =  nfT 


(5-6)  becomes 


2TTTf<- 


.:_2. 


u  I  <  4_  fq  T‘2  sm-u 

'  Rn'  [_^max  J 


+  2P 


max 


-2TTTf. 


00  —f 

/  ?] 

2f„  f 


(Con't. ) 
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tt^T  L 


r^Q  ( 

\  du  ^ 

2nTf„ 

+  f 

du  j 

P  “F 

‘^max 

1  ^max  \ 

^0 

J 

n 

^  / 

^2 

2Qn,ax  ("  +  ln2Tf2'  + 


max 

fo 


(5-7) 


which  is  of  the  order  of  (In  T)/T. 
Next,  for  m  n, 


1 


Tt2t  _  a. 


J  df$(f  +  n/T) 


sin  rrfT 
f(f- 


^  J*  df  [^(f+n/T)  -  $(n/T)] 


r.2rr 

TT  T  —  00 


f(f- 


(5-8) 


since 


J  df  =  0 


f(f-  -2?^) 


for  m  y?  n 


(5-9) 


Proceeding  as  before,  we  have 


nm 


2^2  -2t2 


.Fl 


;  j  +/ 
2 


'-t-2f2  -»  2f2(  ( 


df  l$(f +  n/T)-#(n/T)] 


2 

sin  rrfT 
f(f-  ^))  -i 


tt2t  \L 


^  dfQ  +P 

tt-T  '--2f  If-  ISJIS  I 


-2f2  ,  » 


E 

"  ^  [ 


Q—,.  J  df 


^^2  sin^nfT 


2P, 


max 


max 


-3f. 


|f|  *2 

2P, 


2Q  (n  +  ln3Tf_)  + 
^max  2 


’  “[ 
max 

h  J 


(f+V^  2f2  i^-^y 


(5-10) 
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Tlius,  by  (5-7)  and  (5-10),  for  T  larger  than  any  specified  T^,  we  can  write 


|r  -  6  $(n/T)|  <  K„ 

'  nm  nm  '  '  R  T 


(5-11) 


where  6  is  the  Kronecker  delta  and  K„  is  a  finite  constant  which  depends  on  T  but 
nm  R  o 

is  independent  of  T,  a,  n  and  m.  This  rate  of  convergence  is  sufficiently  rapid  to 
allow  proof  of  the  final  results. 

It  is  also  necessary  to  establish  the  asymptotic  behavior  of 


d  = 


nm 


9  a  ^nm 


1  r  V  sin"  TifT 

tt2t  -  n/T)  (f  -  m/T) 


First  consider 

d  =  ■ 


oo  ^ 

^  J  df  (P'(n/T;ao)  +  P’(f  +  n/T:aQ)-P’(n/T:aQ)] 


n^T  - 


=  P’(n/T;a„)  -f 


(5-12) 


Then 


2f,  -2f, 


1/ 


-T  +  f  1  ■it|P’(f  +  n/T;a„)-nn/T;a„) 

n^T  J_2f2  -00  2f2  j 


sin  nfT 


f^ 


(5-13) 


By  the  assumptions  on  P'(f;a)  and  the  mean  value  theorem  for  the  derivative, 


|P’(f-hn/T:aJ-P’(n/T;a  )1  ^  lf|  Q’ 


max 


(5-14) 
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so  a  development  analogous  to  that  given  previously  shows 


I—  2P*  n 


2 


(5-15) 


An  analogous  development  also  holds  for  m  n,  so  for  T> 


In  T 


|d  -  6  P’(n/T;a  )|  s  K_  _ 
'  nm  nm  o''  D  T 


(5-16) 


where  Kj^  is  a  finite  constant  independent  of  T,  a,  n  and  m. 
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VI.  CONVERGENCE  OF  THE  INVERSE  OF  THE  COVARIANCE  MATRIX  OF  THE 
FOURIER  COEFFICIENTS 

Let  smallest  eigenvalue  of  R  and  let  (R)  ^  be  the  (n,m)  element 

of  R  .  It  has  been  shown  elsewhere  that 


1 


r 

nn 


-  - 
nn 


(6-1) 


where  ^  indicates  that  the  term  in  the  summation  corresponding  to  i  =  n  is  omitted. 
In  Section  V  it  was  established  that  for  T>  T 

o 


<  K 


R 


In  T 
T 


i  n 


(6-2) 


so  with  (3-2) 


i=l 


2 

r. 

in 


<  f  K 
^2*^r 


2  (In  T)' 


(6-3) 


By  Appiendix  A 


so 


where 


X  .  (R)  s  P  . 
min—  min 


^  (R  ^ 


nn 


"  ""  r^n-C  (In  T)^/T 


^2  4 

C  =  ^  >  0 

min 


(A -5) 


(6-4) 


(6-5) 
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Thus,  with  (5-11),  for  T>  T 


1 


$(n/T)  +  K 


^  <s  ^ 


R  T 


$(n/T)  -  K 


In  T  On  T)^ 


R  T 


(6-6) 


For  any  <V'>  0  and  (€|  ^  cp/2 


2|€| 


1  e  1  1 

—  —  —  <  -  <  —  + 

cp  CO  cp  +  €  cp  ^2 


and  therefore 


1 


K. 


- IilX  < 

$(n/T)  p2  T  -  ^nn 


1  2 
+ 


min 


?(n/T)  "  p2 


In  T  ,  ^  (In  T)^ 


1 


min 


(6-7) 


It  can  also  be  shown  that 
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l<^'\ml  "  TI 


N, 


X"  .  (R) 
mm  - 


[|r  |+(X  r.  )/^min(^)l 

‘ '  nm  ^  in '  '  min'—' ' 


i=l 


in'  '  min' 


(6-8) 


With  (A -5),  (6-2)  and  (6-3),  it  follows  that 


|(R~\ml  -  CKr  In  T/T  +  C  (In  T) Vf] 


R 


min 


Combining  this  with  (6-7)  gives  for  T>  Tq 


|(R'S  -  6  $(n/T)|  ^ 

'  —  nm  nm  '  '  ' '  p2 


^R^^  C  (In  T)^ 

T  +  T 


(6-9) 


1 


min 


:£  C 


(In  T)' 
R  T 


(6-10) 
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where  is  a  finite  constant  which  depends  upon  but  not  upon  T,  rt,  n,  or  m.  Thus 
as  T  becomes  large,  the  off-diagonal  elements  of  R'^  converge  to  zero  as  (In  T)^/T. 
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VII.  ASYMPTOTIC  BEHAVIOR  OF  THE  APPROXIMATE  ESTIMATES 


It  will  now  be  established  that 


lim 

T-co 


S(a*) 

S(a) 


1 


(7-1) 


so  that  under  the  criterion  of  smallness  of  the  differential  variance  a*  is  asymptotically 
as  good  as  a.  This  is  accomplished  by  showing  that 

lim  C  =  0  (7-2) 

T  -♦  00 

where 

J*  [  L’  (x;  a)  -  A’  (x;  a)  f(x;  a)  dx 

C  =  — ^ ^ - 

J  [L'(x;a)]  f(x;a)dx 

^num 

^den 

appears  in  (2-15). 

By  (4-7)  and  Appendix  B,  is  of  the  order  of  T.  The  development  which 

follows  is  devoted  to  establishing  that  is  of  the  order  of  (In  T)"^  so  that  asymp¬ 

totically  C  is  of  the  order  of  (In  T)^/T  and  therefore  satisfies  (7-2). 

Let  R  be  the  diagonal  matrix  whose  diagonal  elements  are  =  P(n/T;aQ)  and 
whose  off-diagonal  elements  are  zero.  Let  A  have  diagonal  elements  =  P’(n/T;a^) 
and  off-diagonal  elements  zero.  From  (3-8),  (4-5)  and  (3-9) 

C  =  E(tr  R'^  D-tr  r"^  A  -  r'^  D  r‘^  v  +  R'l  A  R‘^  y 

=  (tr  (R-1  D  -  R‘^  A  )]^  -  2  (tr  (R'^  P  -  R'^  A  )]  (tr  (R'^  D-r'^  A  R’^R). 
+  (tr  (r"^  P  -  r"^  a  R'^  R)  +  2  tr  (R'^  P  "  ^  R)^ 

Con’t. 
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(tr  A  R’l  R A)l^  +  tr  (R'^  D-r‘^  A  R)^ 


‘num  1  ^  ^num  2 


(7-3) 


By  (5-11) 


^niim  1 


V  P’(n/T;cv,)  ,  p,  .ry  .. 
Ij  -5  (r  p  -  P(n/T ;  a  ) ) 

n  P2(n/T;a)  ""  ° 


-i2 


f„  P*  K_  In  T 
2  max  R 
p2~ 

min 


-i2 


(7-4) 


^num  2  hounded  by  observing  that  for  any  square  matrix  A 


tr  s  tr  A  a'^  =  T  -2 


nm 


nm 


(7-5) 


So  if 


lb  1  s  la  I  for  all  n,m 

'  nm  '  nm' 


(7-6) 


then 


.2  Y'  1-2 

tr  A  £  / ,  b 

—  nm 

n,m 


(7-7) 


Thus  we  can  write,  by  (5-11),  (5-16)  and  (6-10), 

^num2  "  I<i  ^  +  +  (ft+Hg)!^ 

=  tr  I R  V,  +  y. j  A  +  U  j  ^  A  R  ^ 


tr  [R  ^  1)2  +  A^  +111  !l2  ^  ^3 


(7-8) 
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"f- 

where  A  has  elements  which  are  the  absolute  values  of  those  of  A,  all  elements  of  T]  , 

Cl ri  T*) ^  In  TP  ^ 

are  equal  to  C  — ■  •  ■  ,  all  elements  of  ti  are  equal  to  K  — :::r"  and  all  elements 


R  T 
of  ri  are  equal  to  K 

K 

Then 


In  T 


-2 


D 


"nun,  2  "2'^) 


KplnT 


P  .  T 
min 


+  P’  C_ 
max  R 


On  T)^ 


+  *^0 

R  D 


_  _,3  F  In  T 

Qn  T)  ^  max  R 

t2  p2  Y 

min 


(7-9) 


From  (7-4)  and  (7-9),  C  is  of  the  order  of  (In  T)  for  T>  T„.  Since  is  of  the 

’  num  '  '  4  o  ’’den 

order  of  T,  we  conclude  that  C  is  of  the  order  of  — ;j; —  or  less  which  establishes 
(7-2)  and  hence  the  desired  result. 
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VIII.  CONVERGENCE  OF  THE  APPROXIMATION  TO  THE  DIFFERENTIAL  VARIANCE 


It  has  been  shown  (3-10)  that 


S(a)  =  {tr  R'^  D  r'^  D]'^ 


(8-1) 


A  convenient  approximation  (4-7)  for  S(a)  is 


/  -E  L’V''-a  ^  P'(n/T;ao)  ^i-l 

i  E  L  (Y.a^n  -  i  L  T(57TTZy  ^ 


=  {tr  R.’^  A  rT^  A}'^ 


(8-2) 


It  was  shown  that  for  T>  T^  this  latter  expression  is  of  the  order  of  1/T.  It  will  now 
be  established  that  as  T-*®®  this  expression  approaches  S(a)  and  in  fact 


lim 

— ♦  oo 


{-E  L*’(y;CXo)}'^_ 

S^) 


(8-3) 


With  the  notation  introduced  in  Section  VII,  (3-10)  and  (4-7) 


-  E  A'’(_^‘,ao) 

-EL’'(v;a^) 


-1  2 
tr  (R  D)^ 

-1  2 

tr  (ft  A) 

tr  [  ^  ^  +  jjj^)  (A  +  ji^) 

n  2 

tr  (ft  A) 


(8-4) 


so 


-E  a'’(y;%) 

-EL'(Y:a^) 


-  1 


tr  (ft'^  +  Hi  4+ Hi  112)^ 

2 

tr  (ft  A) 


(8-5) 


By  the  same  type  of  argument  employed  in  the  previous  Section  it  can  be  shown  that  the 
right  hand  side  of  (8-5)  is  of  the  order  of  (In  T)'^/T  so  that  (8-3)  is  established  and  there¬ 
fore  the  validity  of  the  approximation  of  S(a)  by  {  -EL"(Y;aQ)}  ^  is  confirmed. 
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APPFNDIX  A.  EXTREME  EIGENVALUP:S  OF  THE  COVARIANCE  MATRIX  OF  THE 
FOURIER  COEFFICIENTS 


For  any  vector  x, 


X  (R)  =  max 
max  — 

X 


X 


T 

X  X 


Now 


T  *^2 

R  X  =  y. 


n,m=N 


sin^  TTfT 


1 


'n  ""m  ^2^^^  (f  -  n/T)  (f  “  m/T) 


4-  F  df  $(f)  r V  X 

tt2t  _i  l-n  "  (f-n/T)J 


Since  the  integrand  of  (A -2)  is  non-negative 


T 

X  R  X  s 


p  00 

max  r  Hr 

np 

>  I  1  —  OD  •—  n 


r  V  sin  nfT  ~[' 

I  (f  -  n/T)  I 


max 


TT2'r  ^  n  m 
"  1  n,m  •-<» 


sin  nfT 


^  1  (f  —  n/T)  (f  —  m /T) 


<  P  J  X 
max  ^  n 
n 


and 


Similarly, 


X  (R)  ^  P 
max  —  max 


X  .  (R)  >  P  . 
mm—  min 


Hence,  if  P  .  >0,  R  is  positive  definite, 
min  -  ' 


(A-1) 


(A -2) 


(A -3) 


(A -4) 


(A -5) 
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APPENDIX  B.  THE  APPROXIMATION  OF  SUMS  BY  INTEGRALS 


Assume  that  u(x)  and 


du(x) 

dx 


—  <»<a  ^  X  ^  b<“>  and  that 


are  continuous  (and  therefore  bounded)  for 


b 

I  =  J  u(x)  dx 
a 


exists.  Then 


I 


N-1 

Z 


n=0 


I 

n 


(B-1) 


where 


I 

n 


a+(n+l)A 

=  J  u(x)  dx 

a+nA 


and 


NA  =  (b-a) 


By  the  mean  value  theorem  for  the  derivative 


|u(x)-u(nA)I  ^  Au’  for  nA  <  x  ^  (n  +  1)A  (B-2) 

'  '  max 


where 


u’  =  max 
max  V 


By  the  first  law  of  the  mean 


I  =  Au  (§)  (B-3) 

n 


for  some  such  that 


du(x) 

dx 


for  a  ^  X  ^  b 
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a  +  nA  <  5  <  a  -1-  (n  +  1)A 


(B-4) 


I  lencc 


|l^-Au(nA)| 


max 


A  lu(x)  — u(iiA) 


a  +  nA2:x^a  +  (n  +  1)A 


:£  A'^  u' 


max 


(B-5) 


by  (B-2).  I’lierefore 


N-1 

|I-A  3!  u(nA)l 
n=0 


A"^  u‘  =  A(b  -  a)  u' 
max  max 


(B-6) 


and  the  difference  benvecn  the  integral  and  the  finite  approximating  sum  is  of  the  order 
of  A.  On  the  other  hand 


N-1  b 

^  u(nA)  “  T  J  iK'^)  dx 
n=0  ^  a 


(h  —  a)  u' 


max 


(B-7) 


Tile  right-hand  side  of  (B-7)  is  a  constant  independent  of  N.  Since 


J.  ^ 

A  (b  —  a ) 

the  left-  nrd  side  of  (B-7)  is  of  the  order  of  N  for  N  >  >  0. 
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IS.  ABSTRACT 

An  asymptotic  analysis  is  carried  out  for  an  approximate  method  of  estimating  the 
parameters  of  the  power  spectrum  of  a  zero-mean  stationary  Gaussian  random  process 
from  an  observed  realization  of  limited  duration.  Maximum  likelihood  estimates  are 
obtained  with  the  approximation  that  the  coefficients  of  the  Fourier  series  expan¬ 
sion  of  the  realization  are  uncorrelated.  The  dispersion  of  estimates  is  evaluated 
in  terms  of  a  differential  variance.  With  this  quantity,  the  approximation  estimates 
are  shown  to  be  as  good  asymptotically  as  the  exact  maximum  likelihood  estimates . 

An  approximate  expression  for  the  differential  variance  in  terms  of  the  power  spec¬ 
trum  is  given,  which  is  shown  to  asymptotically  approach  its  exact  value. 

These  results  follow  from  a  general  expression  obtained  in  terms  of  a  converse  to 
the  Schwarz  inequality,  which  compares  the  differential  variances  of  the  approximate 
estimates  and  the  maximum  likelihood  estimates .This  expression  is  evaluated  for  the 
power  spectCTjm  parometer  estimation  problem  in  terms  of  the  covariance  matrix  of  the 
Fourier  coefficients .  The  asymptotic  behavior  of  these  covariances  is  boiinded  to 
demonstrate  the  convergence  of  the  inverse  covariance  matrix  elements.  The  results 
on  the  differential  variance  of  approximate  estimates  are  established  by  matrix 
methods . 
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